The Eigen-Decomposition: Eigenvalues and Eigenvectors

Eigenvectors and eigenvalues are numbers and vectors associated to square matrices, and together they provide the eigen-decomposition of a matrix which analyzes the structure of this matrix. Even though the eigen-decomposition does not exist for all square matrices, it has a particularly simple expression for a class of matrices often used in multivariate analysis such as correlation, covariance, or cross-product matrices. The eigen-decomposition of this type of matrices is important in statistics because it is used to find the maximum (or minimum) of functions involving these matrices. For example, principal component analysis is obtained from the eigen-decompositionof a covariancematrix and gives the least square estimate of the original data matrix. Eigenvectors and eigenvalues are also referred to as characteristic vectors and latent roots or characteristic equation (in German, “eigen”means “specific of” or “characteristic of”). The set of eigenvalues of a matrix is also called its spectrum.